DOCUMENT RESUME 



\ 

\ 



ED 406 426 


TM 026 371 


AUTHOR 


Norton, Scott M. ; Park, Hae'-Seong 


TITLE 


Relationships between Test Preparation and Academic 
Performance on a Statewide High School Exit 
Examination. 


PUB DATE 


Nov 96 


NOTE 


14p. ; Paper presented at the Annual Meeting of the 
Mid“South Educational Research Association 
(Tuscaloosa, AL, November 6“8, 1996). 


PUB TYPE 


Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 


EDRS PRICE 


MFOl/PCOl Plus Postage. 


DESCRIPTORS 


*Academic Achievement ; Chi Square; Correlation; 
English; Ethnic Groups; Graduation; High Schools; 
*High School Students; Mathematics; ^Readiness; 
Sciences ; Sex Differences ; Social Studies ; ^Student 
Reaction; ’'Study Habits; Test Use; Writing 
(Composition) 


IDENTIFIERS 


*Exit Examinations; ^Louisiana 



ABSTRACT 

The relationship between test preparation and 
academic performance on a high school graduation test was studied 
using data from the 1995-96 Louisiana Graduation Exit Examination 
(gee). Test preparation was measured based on students^ responses 
concerning whether they prepared well for the test or not. Academic 
performance was measured through five subjects: English Language 
Arts, Mathematics, Written Composition, Science, and. Social Studies. 
Each subject was examined based on two test uses: students' passing 
status and students' scaled scores. A series of chi square tests were 
used for each subject to examine the relationship between test 
preparation and passing status. In addition, Pearson correlation 
analysis was used for each subject to examine the relationship 
between test preparation and scaled scores. Results show that there 
is a significant relationship between preparation and academic 
performance, but that the magni tudes of the relationships are 
different across the five subjects. The strongest relationship was 
found for mathematics. The patterns of the relationships between test 
preparation and academic performance were generally consistent across 
subgroups with reference to gender and ethnicity. (Contains eight 
tables and seven references.) (SLD) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

Vc ************************ **^ * *************************************** ** 



o 

ERIC 







U.S. DEPARTMENT OF EDUCATION 
Office Qi Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

document has been reproduced as 
received from the person or organization 
originating it. 



□ Minor changes have been made to 

improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERt position or policy. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 

Sctorr H • 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



VO 

(N 

VO 

O 



Q 



w 



Relationships Between Test Preparation and Academic 
Performance on a Statewide High School Exit Examination 



Scott M. Norton 

Louisiana Department of Education 

Hae-Seong Park 
University of New Orleans 



V. 



^ - 
- 




Paper presented at the annual meeting of the 
Mid-South Educational Research Association 
November 1996 
Tuscaloosa, AL 



2 

BEST COPY AVAIMBJLE 



Abstract 



The purpose of this study was to determine the relationship between test 
preparation and academic performance on a high school graduation test. The study 
utilized data from the 1995-96 Louisiana Graduation Exit Examination (GEE). Test 
preparation was measured based on students’ responses concerning whether they 
prepared well for the test or not. Academic performance was measured through five 
subjects: English Language Arts, Mathematics, Written Composition, Science, and 
Social Studies. Each subject was examined based on two usages of the test: students’ 
passing status and students’ scaled scores. 

A series of chi-square (x^) tests were employed for each subject to examine the 
relationship between students’ test preparation and passing status. Also, Pearson 
correlation analysis was employed for each subject to examine the relationship between 
test preparation and students’ scaled scores. The results show that there is a significant 
relationship between students’ preparation and academic performance. However, the 
magnitudes of the relationships were different across five subjects. In particular, the 
strongest relationship was found in Mathematics. The patterns of the relationships 
between test preparation and academic performance were generally consistent across 
subgroups related to gender and ethnicity. 
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Introduction 



There is considerable interest within the education community about student 
achievement, as measured through large-scale standardized tests. Although student 
achievement has been highly studied, little research attempts to show the relationship 
between students’ performance and their perception of their level of preparation. In 
particular, students can fall into four categories: (1) students who do well and think 
they are prepared, (2) students who do well and think they are unprepared, (3) students 
who do not do well and think they are prepared, and (4) students who do not do well 
and think they are unprepared. Of greatest interest to the practitioner should be those 
students in the third category -- those who indicate that they are prepared for the test, 
but in fact, are not. 

As the trend toward large-scale assessment continues to grow, it becomes 
increasingly important to examine possible influences on achievement. This study 
focuses on two particular characteristics of the students: indication of test preparation 
and academic performance, as measured through the pass/fail status and scaled score on 
a Louisiana statewide assessment. The purpose of this study was to examine the 
relationships among these variables. 

Statewide Criterion-Referenced Testing in Louisiana 

Act 40 of the First Extraordinary Legislative Session of 1996 reenacted Revised 
Statute 17:24.4 of the 1986 regular session of the Louisiana Legislature. This 
legislation created the Louisiana Educational Assessment Program (LEAP), which 
includes criterion-referenced testing for grades 3, 5, 7, and the secondary level. The 
State Board of Elementary and Secondary Education (SBESE) has also mandated that 



all high school students be tested under this program as a graduation requirement. 
Because the tests are used as a graduation requirement, they fall into the category of 
known as high-stakes tests (Louisiana Department of Education, 1996a). 

The grades 10 and 1 1 components of the criterion-referenced testing program are 
known as the Graduation Exit Examination (or “Exit Exam”). In grade 10, Louisiana 
students first take tests in English Language Arts, Written Composition, and 
Mathematics. In grade 11, students first take tests in Science and Social Studies. In 
addition to earning 23 Carnegie units, passage of all-five components is required to 
receive a high school diploma. Students who initially fail one or more of the tests 
receive remediation, and retest administrations are offered twice during the eleventh 
grade year and four times during the twelfth grade year (Louisiana Department of 
Education, 1996a). 

Research Questions 

Much of the research concerning large-scale assessment has focused on the 
overall influences on student achievement, or establishing causal-comparative between 
some characteristics and student achievement. Student characteristics and preparation 
have been highly studied, most particularly though the various NAEP assessments 
(e.g.. National Assessment of Educational Progress, 1994a, 1994b, 1994c, 1996a, 
1996b). However, less has been done in the area of examining the students’ perception 
of their preparation. Do students believe that they are prepared? Do they have an 
— understanding of the difficulty level of the test? This study provides information about 
the students’ preparation as related to their overall abilities on the five parts of the 
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Based on this framework, this study was designed to address the following 
research questions: 

1. Is there a relationship between the student’s preparation and their passing 
status on the Graduation Exit Examination? 

2. Is there a relationship between the student’s preparation and their actual 
test performance on the Graduation Exit Examination? 

Methodology 

Sample 

The sample consists of all tenth- and eleventh-grade students who participated in 
the Spring 1996 statewide administration of Louisiana Graduation Exit Examination. 
Students who responded to a sufficient number of items to be assigned a score and 
answered the survey question concerning test preparation were included in the sample. 
Because they make up over 97 % of the public school population, the sample includes 
only African-American and White students. In grade 10, African-American students 
make up 42% of the sample, while White students comprise the remaining 58%. In 
grade 11, African-American students make up 41 % of the sample, while White students 
comprise the remaining 59%. The grade 10 group is 47% male and 53% female; the 
grade 11 group is 46% male and 54% female. For each subject area test, the following 
numbers of students are included: English Language Arts, 43,194; Mathematics, 

42,978; Written Composition, 34,988; Science, 37,969; and Social Studies, 37,957. 
Instrument 

The Graduation Exit Examination provides several different kinds of information 
to the test takers. For promotion and graduation decisions, pass/fail information is 



provided. For diagnostic information, students are provided percent correct 
information (total and domain); scaled scores are also provided for each test taken. 

The English Language Arts, Mathematics, Science and Social Studies tests are 
traditional multiple-choice exams of approximately 60 questions each. The Written 
Composition Test is a performance assessment where students are asked to write an 
essay of no more than two pages on an assigned prompt. Each essay is scored on a 
four-point scale by two raters for each of five dimensions (Louisiana Department of 
Education, 1996a). 

Variables 

Independent variable. The students’ test preparation was measured through the 

use of a survey question on each subject area test. After completing each test, students 

were instructed to respond to the following question: 

“Were you adequately prepared for this test? Think back 
over your entire school experience. In your classes, have 
you had the opportunity to learn the skills necessary to 
complete the (subject name) section of the examination 
successfully? Mark “Y” for “Yes” or “N” for “No” in the 
circle provided on the answer document. This question does 
not count towards your score on the test.” 



Dependent variables. Academic performance was examined through two 
measures: students’ passing status (pass/fail) and scaled score. A passing score 
indicates that a student achieved the performance standard necessary to meet the 
minimum competency level for that test. The scaled score gives a standardized 
accounting of the student’s actual performance on the test (Louisiana Department of 
Education, 1996b). 



This study also uses several demographic characteristics of the students (gender 
and ethnicity). The students’ gender (male, female) was recorded on the answer 
document by the student or the test administrator. The students’ ethnicity (black, 
white) was also recorded on the answer document by the student or the test 
administrator. 

Data Analysis 

The data were analyzed using a two-part approach. First, a series of chi-square 
tests were employed to examine the relationship between test preparation and the 
students’ passing status. In this analysis, students can fall into one of four categories; 
(1) students who passed and indicated they were prepared, (2) students who passed and 
indicated they were not prepared, (3) students who failed and indicated they were 
prepared, and (4) students who failed and indicated they were not prepared. 

Secondly, Pearson correlation analysis was employed for the total group of test 
takers to examine the relationship between test preparation and students’ scaled scores. 
This analysis was employed first for all students, then disaggregated by subgroup 
(African-American males. White males, African-American females. White females). 

Results 

Chi-Square Analysis 

The results show that there is a statistically significant relationship (p < .001) 
between students’ test preparation and academic performance for each subject area test 
on the Graduation Exit Examination. The chi-square (xO tests indicate that the 
independent variable test preparation is significantly associated with the students’ 



passing status in all five subject areas, as shown in Tables 1-5. The significance of 
these associations are primarily a function of the large sample size; however, the phi 
((j)) values show the relative effect size, or magnitude of each association. 



Table 1. Chi-Square Analysis, English Language Arts. 





Failing 


Passing 


Total 


J1 






A 




A 


Test 

Preparation 


“NO” 


1108 


2.6 


2024 


4.7 


3132 


7.3 


“YES” 


4694 


10.9 


35368 


81.9 


40062 


92.8 


Total 


5802 


13.4 


37392 


86.6 


40062 





X%= 1398.45, /7 < .001, (j) = .18 



Table 2. Chi-Square Analysis, Mathematics . 





Failing^ 


Passing 


Total 


J1 


JL 


-tL 


3l. 




A 


Test 

Preparation 


“NO” 


3804 


8.9 


4872 


11.3 


8676 


20.2 


“YES” 


5947 


13.8 


28355 


66.0 


34302 


79.8 


Total 


9751 


22.7 


yilll 


n.2> 


42978 





X^i)=2773.93, /7 < .001, (j) = .25 



Table 3. Chi-Square Analysis, Written Composition. 





Failing 


Passing 


Total 


J1 






A 




A 


Test 

Preparation 


“NO” 


402 


1.2 


2430 


7.0 


2832 


8.1 


“YES” 


1401 


4.0 


30755 


87.9 


32156 


91.9 


Total 


1803 


5.2 


33185 


94.9 


34988 





X^(„ = 515.41, p < .001, (f) = .12 
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Table 4. Chi-Square Analysis, Science. 





Failine 


Passing 


Total 




JL 


JL 


A 






Test 

Preparation 


“NO” 


3151 


8.3 


7896 


20.8 


11047 


29.1 


“YES” 


3846 


10.1 


23076 


60.8 


26922 


70.9 


Total 


6997 


18.4 


30972 


81.6 


37969 





X^,) = 1056.31, p < .001, (j) = .17 



Table 5. Chi-Square Analysis, Social Studies. 





Failing 


' Passing 


Total 


JiL 




JL' 


A 






Test 

Preparation 


“NO” 


1627 


4.3 


8074 


21.3 


9701 


25.6 


“YES” 


2176 


5.7 


26080 


68.7 


28256 


74.4 


Total 


3803 


10.0 


34154 


90.0 


37957 





X^i) =659.04, p < .001, 4> = .13 
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As shown above, the students’ test preparation is significantly associated with 
the students’ passing status in all five subject areas. Because the significance of these 
associations are primarily a function of the large sample size, phi ((j)) was used to show 
the relative effect size, or magnitude of each association. 

The effect sizes for the subjects of English Language Arts, Written Composition, 
Science, and Social Studies are relatively low. However, the effect size for 
Mathematics, in comparison, is slightly higher ((|) = .25). This shows that the 
student’s perception of Mathematics test preparation has a higher relationship to the 
passing status than in the other subjects. In other words, if all tests had an equal 
attainment rate (which they do not), students who answered “YES” to the test 
preparation question in Mathematics would be more likely to pass the test than students 
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who answered “YES” in the other four subject areas. 

From a educational policy viewpoint, these data are useful to show whether 
students can effectively predict their performance on the Graduation Exit Examination. 
For example, students who pass the test would be expected to answer “YES” to the 
survey question; students who fail the test would be expected to answer “NO” to the 
survey question. Students who fail the test but answer “YES” (indicating that they 
were prepared), should be of concern to educators. Students who pass the test, but 
answer “NO” to the survey question misclassify themselves, but are not of as great 
concern due to their passing status. The following table shows the percentage of total 
test takers in each subject who misclassified their passing status. 



Table 6. Percent of Students Misclassifying Their Passing Status. 



Subject 


Percent of Students Who Failed 
and Indicated They Were 
Prepared 


Percent of Students Who Passed 
and Indicated They Were Not 
Prepared 


English Language Arts 


10.9 


4.7 


Mathematics 


13.8 


11.3 


Written Composition 


4.0 


7.0 


Science 


10.1 


20.8 


Social Studies 


5.7 


21.3 



As shown in Table 6, more than 10% of the test takers in English Language 
Arts, Mathematics, and Science all believed they were prepared for the test, but in fact, 
failed the test. These students either overestimate their ability or underestimate the 
difficulty of the test. 

In the grade 1 1 tests of Science and Social Studies, more than 20% of the 
students believed that they were not prepared, but passed the test. These students either 



underestimate their ability or overestimate the difficulty of the test. 

Correlation Analysis 

The results show that there is a statistically significant relationship {p < ,001) 
between students’ test preparation and their actual performance for each subject area 
test, as measured by the scaled score. The Pearson correlation coefficients indicate that 
the independent variable test preparation is significantly associated with the students’ 
scaled scores in all five subject areas, as shown in Table 7. The significance of these 
associations are primarily a function of the large sample size. 

Table 7. Correlation Analysis, Preparation and Actual Performance. 





r 


r* 


P< 


English Language Arts 


.20 


.04 


.001 


Mathematics 


.32 


.10 


.001 


Written Composition 


.18 


.03 


.001 


Science 


.25 


.06 


.001 


Social Studies 


.22 


.05 


.001 



The table above shows the magnitude of the association (r^), or the proportion of 
the students’ actual test performance (scaled score) that can be explained by the 
students’ perception of their test preparation. For four subject areas, the f 
value is low; however about 10% of the students Mathematics scores can be explained 
by their test preparation. 

In some cases, disaggregation by subgroups such as gender and ethnicity reveals 
patterns which are not consistent with the overall findings. However, in this analysis, 
the disaggregated correlation analysis reveals generally the same patterns as the overall 
O subject area correlation analysis. These results are shown below, in Table 8. 

ERIC 
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Table 8. Correlation Coefficients, Preparation and Actual Performance, 
By Ethnicity and Gender. 





Male 


Female 


Total 


African-American White 


African-American White 


English Language Arts 


.20 


.18 


.18 


.16 


.20 


Mathematics 


.27 


.32 


.30 


.33 


.32 


Written Composition 


.18 


.17 


.14 


.12 


.18 


Science 


.21 


.25 


.19 


.21 


.25 


Social Studies 


.21 


.24 


.20 


.20 


.22 



Conclusions . 

In this study, the relationships between students’ test preparation and academic 
ability were examined. While characteristics associated with student achievement have 
been highly studied, this paper attempts to examine the students’ perception of their 
preparation as a predictor of their academic ability. In all subject areas, statistically 
significant associations were found; however, the magnitudes showed that the 
differences are too small to have practical meaning, with one exception. In 
Mathematics, the association was somewhat stronger than in the remaining four subject 
areas. The associations were consistent across two different ethnic groups and also 
between males and females. 

Of more practical significance to the practitioner is the identification of a 
subgroup of students who believe that they are prepared for the Exit Exam, but in fact, 
fail the test. Varying among the different subject area tests, 4 - 14% of the students 
misclassified their passing status. Appropriate and targeted instruction could be 
provided if these students could be identified before the test were administered. 



References 



Louisiana Department of Education (1996a). Louisiana Educational Assessment 
Program Annual Report (Bulletin 1867), Baton Rouge: Author. 

Louisiana Department of Education (1996b). Louisiana Educational Assessment 
Program Technical Manual, Baton Rouge: Author. 

National Assessment of Educational Progress (1994a). NAEP 1992 Reading Report Card 
(Report N. 23-ST06). Princeton, NJ: National Assessment of Educational Progress, 
Educational Testing Service. 

National Assessment of Educational Progress (1994b). NAEP 1992 Mathematics Report 
Card (Report N. 23-ST02). Princeton, NJ: National Assessment of Educational 
Progress, Educational Testing Service. 

National Assessment of Educational Progress (1994c). NAEP 1992 Writing Report Card 
(Report N. 23-WOl). Princeton, NJ: National Assessment of Educational Progress, 
Educational Testing Service. 

National Assessment of Educational Progress (1996a). NAEP 1994 Geography Report 
Card (Report NCES-96-087). Princeton, NJ: National Assessment of Educational 
Progress, Educational Testing Service. 

National Assessment of Educational Progress (1996b). NAEP 1992 Reading Report Card 
(Report NCES-95-045). Princeton, NJ: National Assessment of Educational 
Progress, Educational Testing Service. 



O 

ERIC 



13 





U,s. Department of Education 

Off/ce of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 



REPRODUCTION RELEASE 

(Specific Document) 




I. DOCUMENT IDENTIFICATION: 



Title: i P‘'«Ptt'ro>-isovv (kv\6^ Ac-ac) ■'i vn i' c 

9^yr-fe-fm(x'r>t<s ov\ Sc-Vic5^\ 



Author(s): Sco+4- N\., ^0o^r^T? v- U«-s 

Corporate Source: 



Publication Date: 

ulw 



II. REPRODUCTION RELEASE: 



In order to dsseminate as widely as possible timely and significant materials of interest to the educational community, documents announced 
in the monthly abstract journal of the ERIC system, Resources in Educatiori (RIE), are usually made available to users in microfiche, reproduced 
paper copy, and electronic/optical media, and sold through the ERIC Document Reproduction Service (EDRS) or other ERIC vendors. Credit is 
given to the source of'each document, and, if reproduction release is granted, one of the following notices is affixed to the document 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following two options and sign at 
the bottom of the page. 





The sample sticker shown below will be 


The sample sticker shown below will be 




affixed to ail Level 1 documents 


affixed to all Level 2 documents 


a" 


PERMISSION TO REPRODUCE AND 




PERMISSION TO REPRODUCE AND 




DISSEMINATE THIS MATERIAL 




DISSEMINATE THIS 


♦ 


HAS BEEN GRANTED BY 




MATERIAL IN OTHER THAN PAPER 


SOB 






COPY HAS BEEN GRANTED BY 


Check here 


\0 






For Level 1 Release: 








Permitting reproduction in 








microfiche (4* x 6* film) or 









other ERIC archival media 


TO THE EDUCATIONAL RESOURCES 




TO THE EDUCATIONAL RESOURCES 


(e.g., electronic or optical) 


INFORMATION CENTER (ERIC) 




INFORMATION CENTER (ERIC) 


and paper copy. 










Level 1 


Level 2 




Check here 
For Level 2 Release: 

Permitting reproduction in 
microfiche (4’ x 6’ film) or 
other ERIC archival media 
(e.g., electronic or optical), 
but nor in paper copy. 



Documents will be processed as indicated provided reproduction quality permits. If permission 
to reproduce is granted, but neither box is checked, documents will be processed at Level 1. 



sign 

here-* 

please 




•/ hereby grant to the Educadcnal Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate 
this document as indicated above. Reproduction from the ERIC microfiche or electronic/optical media by persons other than 
ERIC employees and its system contractors requires permi^ion from the copyright holder. Exception is made for non-profit 
reproduction by libraries and other service agencies to satisfy information needs of educators in response to discrete inquiries. * 


Signature: ^ 


Printed Name/Position/Title: 


Org^izatibiVAddre^ 

(VWOv o-f E^Oc.<vViovv 

9.00.. la ToSoA 


tefephbrie: 


E'li^il Address: \ Date: 

SnovVov\@. ri^ov'A. \ .( /'7 Ig, 

\» • 1 ( 1'” 



fover) 



111. DOCUMENT availability INFORMATION (FROM NON-ERIC SOURCE); 

If psrmissior to reproduce is oot gtanled to JtoSmeo* ^(ERK will rat'aooounce a document unless it is 

- - - — - - 



Publisher/Distributor: 




Address: 




Price: 


- 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

,, the ththt to gtant repreduction release is KeM by someone othe, than the addressee, please ptowde the appreptiate name and address: 




Address: 



Ve WHERE TO SEND THIS FORM: 

Sond this form to tha following ERIC Claaringhousa: 

ERIC Acquisitions 

ERIC Clearinghouse on Assessment and Eva;uation 
210 O' Boyle Hall 

The Catholic University of America 
Washington, DC 2006A 

Howevet. « sol^ted by the ERIC Fadlhy, o, I. malung m. unsolicited contnbution to ERIC, return this lotm (and the dooument being 
contributed) to: 

ERIC Processing and Reference Facility 
1100 West Street, 2d Floor 
Laurel, Maryland 20707-3598 



Telephone: 301-497-4080 
Toll Free: 800-799-3742 
FAX; 301-953-0263 
e-mail: ericfac@inet.ed.gov 
WWW: http ://ericfac.piccard. csc.com 



(Rev. 6/96) 



